Papers with layer-wise pruning
Layer-wise Model Pruning based on Mutual Information (2021.emnlp-main)
Copied to clipboard
| Challenge: | In spite of impressive results of neural networks, the huge model size has hindered their applications in cases where computation and memory resources are limited. |
| Approach: | They propose a method for layer-wise pruning using mutual information based feature selection in SVMs and logistic regression. |
| Outcome: | The proposed pruning strategy offers greater speedup and higher performance than weight-based pruning methods. |
VisPCO: Visual Token Pruning Configuration Optimization via Budget-Aware Pareto-Frontier Learning for Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for visual token pruning rely on predefined configurations without determining whether they achieve optimal performance. |
| Approach: | They propose a framework that formulates visual token pruning as a Pareto configuration optimization problem to automatically identify optimal configurations. |
| Outcome: | The proposed framework approximates the empirical Pareto frontier obtained through grid search and generalizes well across pruning methods and VLM architectures. |
Training-free LLM Merging for Multi-task Learning (2025.acl-long)
Copied to clipboard
Zichuan Fu, Xian Wu, Yejing Wang, Wanyu Wang, Shanshan Ye, Hongzhi Yin, Yi Chang, Yefeng Zheng, Xiangyu Zhao
| Challenge: | Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse natural language processing tasks. |
| Approach: | They propose a training-free method for unifying different specialized LLMs into a single model using model-wise and layer-wise pruning and scaling. |
| Outcome: | The proposed method outperforms existing merging techniques and surpasses models fine-tuned on combined datasets in most scenarios. |